Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unified worflow to build TEST BETA and LIVE artifacts #4250

Conversation

gr0vity-dev
Copy link
Contributor

@gr0vity-dev gr0vity-dev commented Jun 22, 2023

This is a draft that builds on our current infrastructure and removes the need for multiple build scripts

Improvements :

  • repository and branch agnostic
  • single workflow to build artifacts for all environments

Behaviour changes

  • no auto builds for the latest release branch
  • remove CI_BUILD flag and allow pre_release version to be set in manual builds
  • workflow succeeds even if no new tag is generated (always green except in case of a real error)
  • Version numbers are taken from the branch it runs on
  • the new tag has the correct CPACK_PACKAGE_VERSION_PRE_RELEASE comitted to CMakeLists.txt

Todos:

  • Executiung the workflow on a release branch should have a slightly different behaviour.

    1. The pre_release_version is not incremented but remains 0
    2. The workflow should increment the minor version by 1 on a subsequent execution
      --> This could be solves by autodetecting if a build already exists.
      --> It could also be solved by an input_parameter (increment_version)
      --> increment_version would increment pre_release for any branch except "releases/" branches
      --> increment_version would increment minor_version in case the workflow operates on "releases/" branch
    3. We want to be able to rerun the workflow without incrementing the Minor version
      --> increment_version could be an integer (0,1,2,-1,...) by which the version is incremented
      --> increment_version could be an boolean (0,1) to enabled or disable incrementing the version
  • Our current workflow fails when no new changes have been committed. This prevents a new tag generation.
    However it would be a better approach to make it pass with a message that no tag was created, so CI remains green.

  • Currently the workflow uses the develop branch to get the MajorVersion number and subtracts 1 in case of a release branch. However this seems fragile. A better approach would be to use the version from the branch the workflow runs on .

  • The pre_release version is calculated during CI and not comitted. This means that even though a Tag is created, building this Tag manually will not result in the same pre_release version as the CI build. Evaluate if we want manual builds from a Tag to have the correct pre_release version.

  • Create a "promotion" workflow that takes a reference, creates a release branch, sets the correct versions (for develop and release) and rebuilds the reference with the new release version numbers. (done in a separate PR)

- Remove CI_BUILD Cmake variable.
- Convert CI_TAG from ENV variable to CMake variable
- CI_VERSION_PRE_RELEASE can be set in non CI builds
- add CI_TAG and CI_VERSION_PRE_RELEASE to build.sh
- add useful ARG with default values to Dockerfile so they can be passed to the build.sh script
- remove CI_BUILD
- convert DCI_TAG from ENV to CMake variable
- Remove dependency on the workflow name
- COnvert $GITHUB_WORKFLOW to $NETWORK
- Create smaller functions with limited scope
- Create similar deploy functions for docker and github container registries (hub.docker and ghcr)
- Remove old workflows
- Create 1 workflow for all environments (Network Matrix)
- Keep current build logic (build scripts still differ per OS)
- skip hub.docker deploys if DOCKER_PASSWORD is not provided
- Create DOCKER_HUB variable which defaults to nanocurrency (backwards compatible)
- Create DOCKER_USER variable which defaults to nanoreleaseteam (backwards compatible)

- create S3_BUCKET_NAME variable that defaults to repo.nano.org if not provided (backwards compatible)
- only use S3_BUILD_DIRECTORY if provided
Convert nanocurrency/nano-env image to self built ghcr.io/${{ github.repository }} image
- convert ref to CI_TAG
- use CI_TAG in aws deploys
@gr0vity-dev gr0vity-dev force-pushed the wip/unified_artifacts_worflow branch from 7ea2a1b to 174df34 Compare June 22, 2023 15:21
gr0vity-dev and others added 18 commits June 29, 2023 16:56
- add possibility to specify registry
- use ghcr.io instead of variable for ghcr_image_name
- vars.S3_BUCKET_NAME
- vars.S3_BUILD_DIRECTORY
- vars.DOCKER_REGISTRY
- vars.DOCKER_USER
The goal is to simplify tag generation process and commit the version_pre_release into CMakeLists.txt for each tag.
If a user checks out a specific tag and builds it, the version_pre_release is set correctly.

- remove workflow_dispatch inputs. It operates on the selected branch.
- The cronjob is executed on develop branch only.
- replace ci/actions/dev-build-tag-gen.sh with ci/actions/generate_next_git_tag.sh
- generate_next_git_tag.sh is branch agnostic and operates on  ${{ github.ref }}
- generate_next_git_tag.sh succeeds even if no new tag is generated
- the workflow only executes the build jobs if a new tag was created (if: ${{ needs.prepare_build.outputs.TAG_CREATED == 'true' }})
- generate_next_git_tag.sh uses V${current_version_major}.${current_version_minor}${branch_name} for tags.
- for "develop" branch_name is converted to DB --> (e.g V26.0DB1)
- generate_next_git_tag.sh uses a -c flag that is responsible to update CMakeLists.txt with correct version_pre_release, create and push the tag to origin.
- use the new prepare scripts (Linux, MacOS & Windows)
- remove the need the dependency on nano-env:gcc
- Build the nano-env docker image in the current workflow
- Use the locally built image.
- Remove duplicate BUILD_TYPE
- move SANITIZER to ci/build-node.sh ARGs
- fixes a bug when we have 2 similar branches with tags (e.g. some-branch and some-branch-2)
Prevent modification of global git settings on a developer machine
- refactored script by making it more modular
- script expects releases to be made from a branch called `releases/v{Major}`
- add -i flag to indicate wether or not to increment the generated tag
- -i increments version_minor for release branches: tag --> V{Major}.{Minor + increment}
- -i increments version_pre_release for all other non-release branches: tag --> V{Major}.{Minor}{branch_name}{pre_release + increment}
- -o outputs either `version_pre_release` or `version_minor` depending on the branch
- make sure V{Major}.0 is created even if the user forgets to set increment to 0
- by redefining local output=$1 it is set to 0 instead of "" and a file called 0 was created
- increment is 1 by default
- if increment is 0, an existing tag will be updated (origin push -f).
@gr0vity-dev gr0vity-dev closed this Jul 3, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant